There is a need to address the problem of enabling the transformation of source code, such as C and C++ program code that includes direct use of C preprocessing directives. Preprocessing can be seen as a pre-compilation meta-programming stage, which, among other things, processes macros and conditionals for different source code branches. There is a need for the ability to correctly handle preprocessing, when applying source code transformation techniques in a commercial software development environment, i.e. beyond academic examples.
FIG. 1 shows an abstraction of this transformation problem. Most powerful source code transformation tools (that can perform context-sensitive modifications) are based on the modification of a structured representation of the source code known as an Abstract Syntax Tree (AST). As the name suggests, the AST provides an abstraction over the lexical detail of source code, while exposing a more generalized structural view. The AST is generated from a target language grammar. In FIG. 1, original C/C++ source code 100 is processed by an AST-based transformation 102 to produce modified C/C++ source code 104.
Existing solutions typically take one of the following two forms, neither of which is practical in a commercial setting. In the first form, transformation is performed after the preprocessing stage (i.e., on pure C and C++ source code) and preprocessing details are not replaced. Loss of preprocessing leads to unwieldy source code that is difficult to maintain and disliked by developers. In the second form, preprocessing directives are partially integrated into the grammar forming a unified grammar by extending the C/C++ languages for typical directive usage. The directive information is parsed and included in the AST. Because of the context-free nature of the C preprocessing language, this is not a complete solution. A significant volume of code is not parsable by the unified-grammar; thus, a pre-transformation stage is required to re-write problematic directive usage.